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(54) Stereoscopic display system and metliod 

(57) According to tlie present invention, tliere is pro- 
vided a stereoscopic display system for displaying a 
model stereoscopically using a pair of images seen from 
two different viewpoints and adapted for the right eye 
and tine left eye comprising gaze range determining 
means(1 3) for determining a range to which a viewer is 
lilcely to fix his or her eyes with respect to the model, 
model move means(1 4) for moving the model in a direc- 
tion perpendicular to a display screen so that the gaze 
range determined by the gaze range determining means 



(1 3) comes in a region in which stereoscopy is easy by 
changing data of the model, model enlargement/reduc- 
tion means (1 5) for enlarging/reducing a size of the mod- 
el moved by the model move means(14) according to a 
distance over which the model is moved by changing 
data of the model changed by the model move means 

(1 4) , and image data creating means(20) for creating 
image data to be displayed on the display according to 
data of model which has been moved by the model 
move means(14) and whose size has been changed by 
the model enlargement/reduction means (15). 
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Description 

Thie present invention relates to a stereoscopic display system, and more particularly to a stereoscopic display 
system which automatically adjusts the stereoscopic display conditions in accordance with changes of a displayed 
5 scene, thereby reducing viewer's discomfort. 

Conventionally, a binocular stereoscopic display system has been l<nown in which images for right and left eyes 
(i.e., images viewed by the right and left eyes, respectively, and hereinafter, referred to as right- and left-eye images 
or right and left images) are displayed certain on a display screen, i.e., a two-dimensional plane, and both the images 
are fused by the viewer's brain Into a single, three-dimensional image. This system has an essential problem of a 
10 discrepancy between accommodation and convergence of eyes. That is, the viewer's eyes are always adjusted so that 
they are focused on the display screen. 

On the other hand, the position of a gazing point on which the viewer's right and left eyes are fixed by eye-ball 
movement (convergence) depends on the distance between the left and right images on a display screen, normally 
parallax; thus, in most cases, that point is not located on the display screen. 
15 When such discrepancy is not so large, humans will have the capacity for fusing the right and left images into a 

single, three-dimensional image When the degree of discrepancy is large, on the other hand, humans will become 
unable to fuse the right and left images, or, they will feel fatigue in their eyes. In such a case, it becomes impossible 
or difficult for them to obtain stereoscopic vision. 

Therefore, users used to adjust the stereoscopic display conditions on a trial and error basis to solve the problem. 
20 This adjustment work is relatively easy for still images. However, for motion images, it is required to change the setting 
of the stereoscopic display conditions for each scene because the position of the viewer's gaze point changes with 
displayed objects of shooting. This is very troublesome. For motion pictures, therefore, the fact is that no change is 
made to the stereoscopic display conditions. 

On the display screen in a stereoscopic display system are often displayed graphical user interface components 
25 such as character information, menus, buttons. However, when these graphical user interface components are not 
adequately displayed in accordance with the chang4e of stereoscopic display conditions, viewers often have strange 
feeling for the Images. 

The recent development of the media like digital versatile disks (DVDs) has realized a system in which viewers 
can freely see the Images from various viewpoints. The function for presenting good stereoscopic Images are required 

30 in such a system. To realize the function, it is strongly desired to solve the above-described problems. 

It is an object of the present Invention to provide a stereoscopic display system which Implements a function of 
automatically adjusting stereoscopic conditions in accordance with scene changes to thereby relieve users of the bur- 
den of complicated operations and prevent the users getting eye fatigue. 

It is another object of the present invention to provide a stereoscopic display system which implements a function 

35 of displaying graphical user interface (GUI) components so that the Image of GUI components match with the model 
to thereby prevent the users getting eye fatigue a feeling of weariness. 

It Is still another object of the present invention to provide a method of creating information for changing stereo- 
scopic conditions in accordance with changing scenes and a storage medium, such as a DVD, which stores the infor- 
mation created by that method together with right-eye and left-eye Image Information. 

40 According to a first aspect of the present invention, there is provided a stereoscopic display system for displaying 

a model stereoscopically using a pair of images seen from two different viewpoints and adapted for the right eye and 
the left eye, comprising gaze range determining means for determining a range to which a viewer is likely to fix his or 
her eyes with respect to the model, model move means for moving the model in a direction perpendicular to a display 
screen so that the gaze range determined by the gaze range determining means comes in a region in which stereoscopy 

45 Is easy by changing data of the model, model enlargement/reduction means for enlarging/ reducing a size of the model 
moved by the model move means according to a distance over which the model is moved by changing data of the 
model changed by the model move means, and image data creating means for creating Image data to be displayed 
on the display according to data of model which has been moved by the model move means and whose size has been 
changed by the model enlargement/reduction means, 

50 According to this configuration, the gaze range Including a gaze point is estimated by the gaze range estimating 

means and the model is moved in the direction of depth so that the gaze range enters the easiness-of-stereoscopy 
region. And the size of the model is changed by the model enlargement/reduction means, according to the distance 
over which it has been moved. The model data thus changed is sent to the image data creating means which, in turn, 
produces image data to be displayed on the display screen. 

55 The system should preferably further comprises depth distance reducing means for setting a scale in the direction 

of depth seen from the viewer so that at least the main part of the model that has been moved by the model move 
means enter the easiness of stereoscopy region. 

According to this, the depth distance is changed so that at least the main(major) part of the model enters the 
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easiness-of-stereoscopy region. 

The system should preferably further comprises out-of-field data removing means for removing data on that portion 
of the model which is situated outside the field of view. 

According to this configuration, since data on that portion of the model which is situated outside the field of view 
5 before the gaze point is estimated is removed, the gaze point can be estimated more quickly and accurately 

The system should preferably further comprises gaze range detecting means for detecting viewer's gase direction 
and point on the images displayed on the display screen, and wherein the model move means moves the model in a 
direction normal to the display screen on the basis of the gaze point detected by the gaze range detecting means. 

According to this configuration, since the stereoscopic conditions are calculated on the basis of the gaze point on 
TO actually displayed images, more suitable stereoscopic conditions can be calculated according to the personality of the 
viewer. 

The system should preferably further comprises gaze range detecting means for detecting viewer's gaze direction 
and point on the images displayed on the display screen, and the out-of-field data removing means should preferably 
calculate a field of view on the basis of the gaze direction detected by the gaze range detecting means and remove 
15 data for the outside of the detected field of view. 

According to this configuration, since the visual field is calculated on the basis of visual lines on actually displayed 
images, a model situated outside the field of view can be determined more suitably according to the viewer's personality 
and model data for the outside of the field of view can be removed. Thus, the gaze point can be estimated more 
accurately. 

20 The system should preferably further comprises tone change means for changing the tone of the model displayed 

on the display screen so that the further away from the viewpoint, the closer the tone becomes the background tone. 

According to this configuration, since the tone of the model is displayed so that the further away from the viewpoint, 
the closer the model tone becomes the background tone, the effect of seeing objects through air is achieved, providing 

more effective stereoscopy 

25 The model moving means should preferably include means, when a change occurs in the placement conditions 

of the model into the three-dimensional space seen from the viewer and the model moving means moves the model 
to fit the change, for preventing the amount of movement of the gaze range in the model exceeding one degree in 
terms of angle of convergence for a fixed period of time subsequent to the occurrence of the change in the placement 
conditions. 

30 According to this configuration, since the amount of movement of the gaze point is limited for a predetermined 

period of time, viewer's eyes fatigue is reduced. 

The image data creating means should preferably have a function of determining stereoscopic conditions of graph- 
ical user interface components to be displayed on the display screen. 

According to this configuration, since not only a model but also graphical user interface components are displayed 
35 stereoscopically the viewer can be relieved of a feeling of strangeness resulting from mixture of three-dimensional 
images and two-dimensional images. 

The system of the first aspect should preferably have means for varying a stereoscopic condition to diminish the 
stereoscopic effect. 

The system shouid preferably further comprising means for sinading-off/blurring the edges of right and left image 

40 of the model, where the model is situated at the edge of the display screen. 

According to a second aspect of the present invention there is provided a stereoscopic method for displaying a 
pair of picture images seen from two different viewpoints with respect to a model on a display screen stereoscopically, 
comprising: gaze range determining step for determining a range to which a viewer is likely to pay attention with respect 
to a model; a model moving step for moving the model in a direction perpendicular to a display screen so that the gaze 

45 range determined by the gaze range determining step comes in a region in which stereoscopy is easy by changing 
data of the model; a model enlarging/reducing step for enlarging/reducing a size of the model moved by the model 
moving step according to a distance over which the model is moved by changing data of the model changed by the 
model moving step; and a data creating step for creating image data to be displayed on the display according to data 
of model which has been moved by the model moving step and whose size has been changed by the model enlarging/ 

50 reducing step. 

According to this configuration, the gaze range including a gaze point is estimated by the gaze range estimating 
step and the model is moved in the direction of depth so that the gaze range enters the easiness-of-stereoscopy region. 
And the size of the model is changed by the model enlarging/reducing step according to the distance over which it has 
been moved. The model data thus changed is sent to the image data creating step which, in turn, produces image 
55 data to be displayed on the display screen. 

According to a third aspect there is provided a storage medium for storing computer programs for dispiaying a pair 
of picture images seen from two different viewpoints with respect to a model on a display screen stereoscopically in 
computer system, comprising: a gaze range determining instruction means for giving instructions for determining a 
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range to which viewers are lil^ely to pay attention to the computer system; a model moving instruction means for giving 
instructions for moving the model in a direction perpendicular to a display screen so that the gaze range determined 
by the gaze range determining means comes in a region in which stereoscopy is easy by changing data of the model 
to the computer system; a model enlarging/reducing instruction means for giving instructions for enlarging/reducing a 

5 size of the model moved by the model move means according to a distance over which the model Is moved by changing 
data of the model changed by the model move means to the computer system; and an image data creating instruction 
means for giving instructions for creating image data to be displayed on the display according to data of model which 
has been moved by the model move means and whose size has been changed by the model enlargement/reduction 
means to the computer system. 

10 According to this configuration, the gaze range including a gaze point is estimated by the computer system and 

the model is moved in the direction of depth so that the gaze range enters the easiness-of-stereoscopy region. And 
the size of the model is changed by the model enlarging/reducing Instruction according to the distance over which it 
has been moved. The model data thus changed Is sent to the Image data creating step which, In turn, produces image 
data to be displayed on the display screen. 

15 According to a forth aspect of the present invention, there is provided a stereoscopic display system for displaying 

a model stereoscopically using a pair of images seen from two different viewpoints, comprising: gaze range determining 
means for determining a range to which a viewer is likely to pay attention with respect to the model; a perspective 
transformation determining means for determining a perspective transformation method of the model data to the pair 
of Images for the left eye and the right eye according to an output of the gaze range determining means; and an image 

20 data creating means for creating the image data to be displayed on the display according to a perspective transformation 
method determined by the perspective transformation determining means. 

According to this configuration, image data easy to view stereoscopically can be produced by the perspective 
transformation without actually moving and enlarging/reducing the model. 

According to a fifth aspect of the present invention, there is provided a stereoscopic display method for displaying 

25 a model stereoscopically using a pair of images seen from two different viewpoints, comprising: a gaze range deter- 
mining step for determining a range to which a viewer is likely to pay attention with respect to the model; a perspective 
transformation determining step for determining a perspective transformation method of the model data to the pair of 
Images for the left eye and the right eye according to an output of the gaze range determining means; and an image 
data creating step for creating the image data to be displayed on the display according to a perspective transformation 

30 method determined by the perspective transformation determining step. 

According to this configuration, image data easy to view stereoscopically can be produced by the perspective 
transformation without actually moving and enlarging/reducing the model. 

According to the sixth aspect of the present invention, there Is provided a storage medium for storing computer 
programs for displaying a model stereoscopically using a pair of images seen from two different viewpoints, comprising: 

35 gaze range determining instruction means for giving instructions for determining a range to which a viewer is likely to 
pay attention with respect to the model to the computer system: a perspective transformation determining instruction 
means for giving instructions for determining a perspective transformation method of the model data to the pair of 
images for the left eye and the right eye according to an output of the gaze range determining instruction to the computer 
system; and an image data creating instruction means for giving instructions for creating the image data to be displayed 

40 on the display according to a perspective transformation method determined by the perspective transformation deter- 
mining instruction. 

According to this configuration, Image data easy to view stereoscopically can be produced by the perspective 
transformation without actually moving and enlarging/reducing the model. 

According to a seventh aspect of the present invention, there is provided a stereoscopic display system for dis- 
45 playing a pair of Images seen from two different viewpoints stereoscopically, comprising: gaze area determining means 
for determining a range to which a viewer is likely to pay attention; an image conversion determining means for deter- 
mining a method for image conversion so that the gaze area becomes a region in which stereoscopy is easy according 
to an output of the gaze area determining means; and an image data creating means for creating the image data to 
be displayed on the display according to an output of the Image conversion determining means. 
50 According to this configuration, the system can be applied to stereoscopic display of actually shot pictures and 

image data that allow the actually shot pictures to be easily viewed stereoscopically can be produced. 

According to an eighth aspect of the present Invention, tiiere is provided a stereoscopic display method for dis- 
playing a model stereoscopically using a pair of images seen from two different viewpoints, comprising: gaze area 
determining step for determining a range to which a viewer is likely to pay attention; an image conversion determining 
55 step for determining a method for image conversion so that the gaze area becomes a region in which stereoscopy is 
easy according to an output of the gaze area determining step; and an image data creating step for creating the image 
data to be displayed on the display according to an output of the image conversion determining step. 

According to this configuration, the system can be applied to stereoscopic display of actually shot pictures and 
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image data that allow the actually shot pictures to be easily viewed stereoscopically can be produced. 

According to a ninth aspect of the present invention, there is provided a storage medium for storing computer 
programs for displaying a pair of images seen from two different viewpoints stereoscopically in computer system, 
comprising: gaze area determining instruction means for giving instructions for determining a range to which a viewer 
5 is likely to pay attention to the computer system; an image conversion determining instruction means for giving instruc- 
tions for determining a method for image conversion so that the gaze area becomes a region in which stereoscopy is 
easy according to an output of the gaze area determining instruction: and an image data creating instruction means 
for giving instructions for creating the image data to be displayed on the display according to an output of the image 
conversion determining means. 
10 According to this configuration, the system can be applied to stereoscopic display of actually shot pictures and 

image data that allow the actually shot pictures to be easily viewed stereoscopically can be produced. 

According to a tenth aspect of the present invention, there is provided an image data storage medium that is 
adapted for use with a stereoscopic display system adapted to display images on a display screen and furnish separate 
images to each eye of a viewer to thereby provide stereoscopy Tine image data storage medium has a storage section 
1S for storing an image conversion method that allows a gaze area of images on which the viewer fixes his or her eyes 
to be positioned in an easiness-of-stereoscopy region for each scene. 

According to this configuration, the storage medium can provide image data that allow easy stereoscopy by being 
applied to the stereoscopic display system of the present invention. 

This invention can be more fully understood from the following detailed description when taken in conjunction with 
20 the accompanying drawings, in which: 

FIG. 1 is a schematic illustration of a stereoscopic display system according to a first embodiment of the present 

invention; 

FIG. 2 is a block diagram of the stereoscopic display system of FIG. 1 ; 
25 FIG. 3 shows a coordinate system used In the stereoscopic display system; 

FIGS. 4A and 4B form a flowchart for the stereoscopic condition calculation processing in the first embodiment; 
FIG. 5 shows the operation of the coordinate transformation means; 

FIGS. 6A through 6D show the flow of processes in the stereoscopic condition calculating section; 
FIG. 7 shows a method of estimating a gaze point on which right and left eyes are fixed; 
30 FIG. 8 shows a method of estimating the gaze point; 

FIG. 9 is a graph showing a relationship between the depth distance between the display screen and the viewpoint 

center and the easiness (possibility)-of-stereoscopy region; 

FIG. 10 is a graph illustrating a method of correcting the interocular distance; 

FIGS. 11 A and 11 B show changes in the amount that right and left images are offset with respect to each other 

35 on the display screen when the interocular distance is corrected; 

FIGS. 12A and 12B show a method of creating the graph of FIG. 10: 
FIGS. 13A and 13B show the operation of the Image data creating means; 

FIGS. 14A, 14Band 14C show the reduction of distance of depth when the interocular distance is corrected; 
FIGS. ISA and 15B show the operation when the entire model Is moved; 
40 FIG. 16 is a block diagram of a stereoscopic display system according to a second embodiment of the present 

invention; 

FIGS. 17Aand 17B form a flowchart for the stereoscopic condition calculation processing in the second embodi- 
ment; 

FIGS. ISA and 1 SB is a graph showing a process to relax the stereoscopic conditions with time in the first and 
45 second embodiments; 

FIG. 1 9 is a block diagram of a stereoscopic display system according to a third embodiment of the present inven- 
tion; 

FIGS. 20A and 20B form a flowchart for the stereoscopic condition calculation processing in the third embodiment; 
FIG. 21 is a block diagram of a stereoscopic display system according to a fourth embodiment of the present 

so Invention; and 

FIGS. 22A and 22B form a flowchart for the stereoscopic condition calculation processing in the fourth embodiment. 

Before describing the embodiments of the present invention, the principles of stereoscopic display will be described 
below. 

ss FIG. 9 is a graph showing a relationship between the depth distance between the viewpoint center (refers to the 

midpoint of the line connecting the viewer's right and left eyes) and the display screen and the possibility -of -stereoscopy 
region (easiness-of-stereoscopy region) expressed in terms of the distance from the viewpoint center (see "THREE- 
DIMENSIONAL DISPLAY" Physiological Engineering 15 by Toyohiko Hatada in the technical journal "O plus E"). 
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Thepossibility-of-stereoscopy region ortheeasiness-of-stereoscopy region is a region tliat makes tlie stereoscopic 
view of images possible or easy (in the direction of deptfi). 

Here, the distance between the viewpoint center and the display screen corresponds to the depth distance between 
the viewpoint center and a point at which both the eyes are focused. The area of the possibility (easiness)-of-stereos- 
5 copy region indicates the allowable range of variations in angle of eye convergence. As can be understood from FIG. 
9, the smaller the distance between the viewpoint center and the display screen, the narrower the possibility (easiness)- 
of-stereoscopy region becomes. 

Consider here the case where the binocular stereoscopy display system is applied to a CAD system that simulates 
a three-dimensional model. In this case, suppose that the distance between the viewpoint center of the system user 
10 and the display screen is 60 cm. Then, the possibility-of-stereoscopy region extends in the depth distance from the 
viewpoint center from An2 (cm) to Af2 (cm). The easiness-of-stereoscopy region, i.e., the region in which stereoscopic 
vision is allowed with little eye fatigue, is limited to a range from Ani (cm) to Afl (cm) in the depth distance from the 
viewpoint center. 

However, the model (an object of display) is not always located between Ani and Afl. When the model is not 

15 located between An1 and Afl, it is required to change the stereoscopic conditions so that the entire model will enter 
the region in which the stereoscopic image is easy to view. 

On the other hand, in the case of usual images with no three-dimensional position data (actually shot pictures such 
as photographs or video pictures), the region where stereoscopic images are easy to view are expressed in terms of 
a critical value of the amount that right and left images are offset with respect to each other. According to a literature 

20 (The GrystalEyes Handbook: LLipton, StereoGraphics Corp. 1 991 ), the critical value is on the order of 1 .5 degrees in 
terms of visual angle. It will not exceed two degrees at a little higher estimate. In making an image perceived behind 
the display screen, the maximum amount that the right and left images are offset corresponds to the Interocular distance. 
If, therefore, the viewpoint center is one meter or more away from the display screen, all the models (objected display) 
behind the screen will be in the easiness-of-stereoscopy region. In making an image perceived before the display 

25 screen, this condition will not be met if the image protrudes, to some degree, from the display screen. As with the three- 
dimensional model, therefore, in order to make a stereoscopic image easy to view, it is required to appropriately change 
the offset amount of right and left images and change the stereoscopic conditions so that the offset amount will always 
become less than the critical value. 

However, the conventional stereoscopic display system is not equipped with means of controlling the stereoscopic 

30 conditions In accordance with changes in displayed images. For this reason, the cinematographer has to shoot images 
while taking subsequent easier viewing into consideration, and the viewer himself cannot change Images on an Inter- 
active basis. 

The present invention performs functions of controlling the stereoscopic conditions on the basis of information on 
gaze points(fixing point/range) for scenes and generating stereoscopic images that is easy for viewers to see. 
35 Hereinafter, the preferred embodiments of the present invention will be described with reference to the accompa- 

nying drawing. 

Reference will now be made to Fl G. 1 to give the outline of the stereoscopic display system of the present invention. 
Hereinafter, the stereoscopic display system is supposed to be of a time-multiplexing type. 

FIG. 1 shows a state in which a left -eye picture image 7a and a right-eye picture image 7b are displayed simulta- 
40 neously on a display screen (which may be a normal CRT). The left picture image 7a consists of a left background 8a 
and a left model image 9a on the background. Likewise, the right picture image 7b consists of a right background 8b 
and a right model image 9b on the background. 

Actually, the left picture image 7a and the right picture image 7b are displayed alternately at a high speed; thus, 
the viewer will recognize one picture image 7, one background 8 and one image 9 as being displayed three-dlmen- 
45 sionally. 

To the display unit 1 Is connected an operations unit 5 for producing Image signals, which calculates the stereo- 
scopic conditions of the image 9, processes left and right image data on the basis of the calculated stereoscopic 
conditions, and provides to the display unit 1 the resulting left and right picture images 7a and 7b alternately at regular 
intervals. 

50 To the operations unit 5 are connected to input means 6, such as a keyboard 6a, a mouse 6b, and a joystick 6c, 

which are adapted to provide commands to rotate, move and designate the coordinates of an Image displayed on the 
screen la of the display unit 1 . 

The stereoscopic system includes liquid-crystal shutter eyeglasses 2, which are adapted to block light directed 
onto each of the right and left eyes alternately in synchronization with switching between the left and right picture 
55 images 7a and 7b. That is, the viewer's left eye 3a views only the left image 7a of the image 7 displayed on the display 
screen la via the left lens 2a of the liquid-crystal shutter eyeglasses 2, while the right eye 3b views only the right image 
7b in the image 7 displayed on the display screen la via the right lens 2b of the eyeglasses 2. 

If the proper stereoscopic conditions have been set up, the viewer will be able to perceive a stereoscopic image 
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by fusing the left and right model images 9a and 9b in the brain. In this case, the distance Za between the viewpoint 
center and the center 1 b of the display screen la is set constant (approximately 60 cm). The distance Za may vary with 
viewers, depending on their positions. In this specification, the following description will be given on the assumption 
that the distance Za is constant. 
5 Hereinafter, an example of a configuration of the stereoscopic display system will be described with reference to 

FIG. 2. 

As shown in FIG. 2, the system has a control means 10 that performs control of the whole system. The control 
means receives commands entered from the input means 6 and produces and issues instructions to the system. 
To the control means 10 is connected a recording means 11 which records information about the shape, size and 
10 arrangement of a model present in three-dimensional space. The control means receives such model Information from 

the recording means. 

To the control means 1 0 Is connected a GUI data manage means 21 that manages data on graphical user Interface 
components such as character Information, menus, buttons, and the like. The control means receives such data from 

the GUI data manage means. 

'5 The system has a stereoscopic condition calculating section This section comprises: an out-of-field data exclude 

means 1 2 that excludes data on a model present outside the field of view; a gaze range determination means 1 3 that 
estimates a gaze range (including a gaze point) over which the left and right eyes are fixed on the basis of model data; 
a model move means 1 4 that performs a process of moving the model in the direction of depth; a model enlarge/reduce 
means 1 5 that performs a process of enlarging or reducing the model; a depth distance reduce means 1 6 that sets up 

20 the scale of the model space in the direction of depth so that the main portion of the model will be included in the 
easiness-of-stereoscopy region; a tone change means 1 7 that changes the tone of the model to impart to the displayed 
image an effect of seeing objects through air; a coordinate system transform means 18 that performs a process of 
putting a model placed in the coordinate system of three-dimensional space into the coordinate system for the stere- 
oscopic display system (stereoscopic coordinate system); an image offset adjust means 22 that adjusts the offset 

25 amount of the right and left images to eye fatigue; and an image edge process means 23 that performs a process of 
shadlng-off/blurring the edges of the display screen. 

The out-of-fleld data exclude means 12, the gaze range determination means 13, the model move means 14, the 
model enlarge/reduce means 15, the depth distance reduce means 16, the tone change means 17, the coordinate 
system transform means 1 8, the Image offset adjust means 22 and the Image edge process means 23 are connected 

30 to the model data manage means 19, which is responsive to Instructions from the control means 10 to send data 
received from those means 12 through 18 to an Image data create means 20. 

The image data create means 20 creates right and left image data on the basis of the model data received from 
the model data manage means 1 9 and sends it to the display 1 . 

The means 10 through 21 are implemented by the operations units in the stereoscopic system. A computer system 

3S may be installed with a computer program that describes procedures that implement the means 1 0 to 21 . 
Next, the operation of the present embodiment will be described. 

Before describing the operation specifically, the coordinate system of the stereoscopic system used In this spec- 
ification will be described with reference to FIG. 3. As shown in FIG. 3, the coordinate system of the stereoscopic 
system is represented by the XYZ coordinate system with the midpoint O between left and right eyes 3a and 3b taken 

40 as the origin. 

In this coordinate system, the X axis is coincident with the line connecting the left and right eyes 3a and 3b and 
extends in parallel with the display screen la of the display unit 1 . The positive direction is taken in the direction of the 
left eye 3a. The Z axis Is coincident with the line connecting the origin O and the center lb of the display screen la. 
The direction from the origin O to the center 1 b is taken as positive. The Z axis is aligned with the center of visual lines. 

45 The Y axis is normal to the XZ plane. The positive direction Is coincident with the up direction. The length of the 

line connecting the left eye 3a and the right eye 3b is termed the interocular distance, and the midpoint of the line 
connecting the left and right eyes, i.e., the origin O, Is termed the viewpoint center. 

The operation of the present embodiment will now be described with reference to a flowchart of FIGS. 4A and 4B. 
As shown In FIG. 4A, at system startup, Initial values which are usually stored in the recording medium 11, are 

50 entered by the input means 6 into the control means 10 (step SI 00). Initial values include an initial value of the inte- 
rocular distance which is used as the criterion for later determining the offset amount of the left and right images 7a 
and 7b, data as to how to place a model placed on a coordinate system of three-dimensional space on the stereoscopic 
coordinate system, the distance Za (= 60 cm) between the viewpoint center O and the center 1 b of the display screen 
la, and the like. The initial interocular distance is set to the human average interocular distance, i.e., 6 cm. 

ss On termination of entry of the initial values, the control means 1 0 reads model data associated with the coordinate 

system in three-dimensional space from the recording means 11 (step SI 01 ) and sends it to the model data manage 
means 1 9 together with the initial values. The model data include the dimensions and shape of the model, the placement 
state of the model in the coordinate system of three-dimensional space, and tones of the model (including all of hue. 
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lightness, and saturation; this remains unchanged throughout the specification). 

Next, the model data manage means 1 9 sends the initial values and the model data to the coordinate system 
transform means 18, which, in turn, places the model in the stereoscopic coordinate system as shown in FIG. 5 based 
on the initial values and the model data (step SI 02). 
5 Next, data are entered from the input means 6 through the control means 1 0 Into the model data manage means 

19 so as to adjust the position and posture of the model that is placed in the stereoscopic coordinate system (step 
SI 04). The posture of the model means a direction that it faces. 

The model data manage means 19 next sends data on the model placed in the stereoscopic coordinate system 
to the out-of-field data exclude means 12, which, of the received model data, excludes data for models (Indicated by 
10 broken lines in FIG. 6A) that exist outside the field of view (step 8105). The resulting model data Is returned to the 
model data manage means 1 9 to rewrite the original model data In it. 

The model data manage means 19 sends the model data to the gaze range determination means 13, which, on 
the basis of the placement conditions of the model in the three-dimensionai space and the initial values received from 
the model data manage means 19, estimates the gaze range in the direction of depth of viewer's eyes on the model 
15 placed in the stereoscopic coordinate system (step S106). In this embodiment, the estimation of the gaze range may 
be performed by either of two approaches discussed below. 

In the first approach, as shown in FIG. 7, boundary boxes Ri (1 = 1 , 2,.., N) are first set up each of which envelops 
a corresponding one of elements Ei contained in the model. 

Next, weights Wi of representative points of elements Ei enveloped by the boundary boxes Ri are sought with each 
20 boundary box as a unit. In seeking the weights, the number of vertexes of each boundary box Ri is calculated first. 
The number of vertexes of a boundary box is normally eight. For a boundary box RI that resides across the boundary 
of the field of view as shown in FIG. 7, however, only the number of vertexes that reside inside the field of view (indicated 
by black dots) is calculated. 

Next, a coefficient that represents the complexity of the element Ei is calculated. When each element is a structured 
25 geometric model element, the coefficient is calculated from the number of sides or surfaces of the element Ei. When 
each element is an image, the coefficient is calculated from spatial frequencies. The coefficient is multiplied by the 
number of vertexes of the boundary box Ri, thereby calculating the weight Wi for each boundary box. 

Next, the coordinates xl, yl and zl of the representative point of the boundary box RI are sought. The Z-axIs coor- 
dinate of the representative point, Zl (of the coordinates xl, yl and zl) Is defined as the distance between the element 
30 Ei and the viewpoint center. The coordinates of the representative point of the element Ei are taken as the coordinates 
of the center of gravity of the element Ei. 

Next, the distance between the representative point of the boundary box Ri and the visual-line(visual-line) center 
(the Z axis of the stereoscopic coordinate system) di is calculated. 

Next, using the weights Wi of the representative points of the boundary boxes Ri, the distances Zl between the 
35 representative points and the viewpoint center, and the distances di between the representative points of the boundary 
boxes Ri and the Z axis, the coordinate Zd of the gaze point in the direction of depth is calculated by 



Y,Zi X Wi X di 



i = l 



2; Wi X di 
i=l 



(1) 



where N is the number of the boundary boxes, Zi is the coordinate of the representative point of the boundary 
box Ri in the direction of depth , Wi is the weight of the boundary box Ri, and di is the distance between the representative 
point of the boundary box Ri and the Z axis. 

The standard deviation of the depth coordinates of the representative points is calculated by 



55 
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2^ {(Zi - Za)2 X Wi X di> 
i = l 



(2) 



NX 2; (Wi X di) 
i = l 



Using the depth coordinate Zd of the reference point, the standard deviation , and the minimum vaiue ZiO and 
maximum vaiue Zil of Zi, the gaze range is caicuiated to be 

IS 

max(Zio,(Zd -axa))~ min((Zd + a X a), Zil) (3) 



where max(a, b) is a function that returns a greater one of a and b and min(a, b) is a function that returns a smaiier 
one of a and b. 

In equation (3), a is a real coefficient and usualiy talces a vaiue in the range from 0 to 2. 

In the second approach, the voronoi poiyhedrons based o representative points Tij are first created in the boundary 
box Ri, which is shown in FiG.8. The Voronoi poiyhedron is for dividing space in the boundary box Ri into the foliowing 
regions in reiation to a given number (= n) of points (the case where the regions associated with points Tlj (j = 1 , 2,.., 
n) are represented by Vi(Tij)). 

Vi (Tij) = {T I d(Ti,Tij) < d(Ti,Tik)} (k=1n) 

where Ti indicates a point inside the boundary box Ri and d(Ti, Tij) indicate the distances between the points Tij 
and Ti. 

That is, in the boundary box Ri, the point Ti inciuded in the region Vi (Tij) is closer to Tij than any other representative 
point. 

Next, a coefficient that represents the compiexity of the element Ei enveloped by the divided regions VifTij) is 
caicuiated. This coefficient is caicuiated from the number of edges or surfaces of the element Ei enveloped by the 
regions VI(TI]) when each element is a structured geometric model element. When each element is represented only 
on an image, the coefficient is caicuiated from spatial frequencies. The coefficient is used as the weight Wij of each of 
the regions 

Next, the distance Zij of the representative point Tij from the viewpoint center is defined by the Z-axis coordinate zij. 

Next, the distance dij between each representative point and the visual-line(visual iine) center (the Z axis of the 
stereoscopic coordinate system) is caicuiated. 

Next, using the weights Wij of the representative points Tlj of the boundary box Ri, the distances Zi between those 
representative points and the viewpoint center O, and the distances dij between the representative points Tij of the 
boundary box Ri and the visual-line center (Z axis), the coordinate Zd of the gaze point in the direction of depth is 
calculated by 



N 9 

2 • 2 (Zij X X dij) 



N 9 

2: • 2^ (Wij X dij) 

i=l j=l 
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where N is the number of the boundary boxes, Zij is the coordinate of the representative point of the j-th region 
of the boundary box Ri in the direction of depth, Wij is the weight of the j-th region of the boundary box Ri, dij is the 
distance between the j-th region of the boundary box Ri and the Z axis, and i indicates the i-th boundary box Ri. 
The standard deviation is calculated by 



(5) 



N 9 






X Wij X dij } 


i=l j=l 




N 9 




NX Z ■ Z 


(Wij X dij) 


i=l j=l 





Thus, the gaze range is calculated in terms of Zd, o, ZiO, and Zil as follows: 

max(Zio,(Zd - aX o)) ~ min((Zd + a X o), Zij) (6) 
In equation (6), a Is a real coefficient and usually takes a value In the range from 0 to 2. 

In the above expressions, the weight W and the distance d between the representative point and the visual-line 
center are multiplied together. Alternatively, use may be made of an expression such that axW + bxd(a and b are 

each a coefficient). 

In both the first and second approaches, Ri is assumed to be a boundary box that envelops a model element Ei 
(i = 1 , 2,.., N). Alternatively, the boundary box Ri may be assumed to be one that envelops part of the element Ei, for 
example, its surface that is visible from the viewpoints. 

In the following, description is given of a method of moving the model with respect to the reference point of the 
gaze range (hereinafter referred to as the gaze point), i.e., with respect to a = 0, by way of example. 

The coordinate of the gaze point in the direction of depth, Zd, obtained by the first and second approaches, is sent 
to the model data manage means 1 9 and stored in it. 

Next, the model data manage means 19 passes the model data and the coordinate Zd of the gaze point in the 
direction of depth to the model move means 14. The model move means then makes a decision as to whether an 
image to be displayed this time, i.e., an image that is a current object of the above-described processing, is a new 
Image or a changed image (step SI 07). The changed image refers to an Image obtained by simply rotating the Image 
displayed just before or changing Its posture (step SI 04), that Is, an Image that has continuity with the immediately 
preceding image. The new image refers to any other image. 

When the image is a new image, the model move means 14 parallel moves the entire model in the direction of 
depth (in the direction of the Z axis) as shown in FIG. 6C so that the gaze point P will enter the easiness-of-stereoscopy 
region shown in FIG. 8, on the basis of the Z-axis coordinate Zd of the gaze point in the direction of depth calculated 
by the gaze range determination means 13 (step S110). In this embodiment, since the distance between the viewpoint 
center O and the center of the display screen 1a is 60 cm, it is necessary that the gaze point be located in the range 
of Ani cm to Afl cm in the depth distance from the viewpoint center O indicated by the abscissa of the graph of FIG. 9. 

When the current image is a changed image, the model move means 14 makes a decision as to whether, on the 
assumption that the process in step S110 was performed at it was, the depth position of the gaze point will move one 
degree or more in terms of angle of convergence relative to the depth position of the gaze point for the image that 
immediately precedes the image that is currently to be processed (step S10G). When the decision is less than one 
degree, the process in step S110 is performed as with the new image. The reason why the criterion for decision Is set 
to one degree is that the movement of the gaze point of one degree or more in terms of angle of convergence causes 
severe viewer's eye fatigue. Since the degree of eye fatigue vary from viewer to viewer, the criterion for decision may 
be set to a value less than one degree, e.g., 0.5 degrees. 

When the decision is more than one degree, on the other hand, the model move means 14 adds a predetermined 
limitation to the movement of the model. That is, in this case, the model move means 14 parallel moves the entire 
model in the direction of depth (Z axis) so that the gaze point moves as close to the easiness-of-stereoscopy region 
as possible while imposing a limitation such that the gaze point after movement is not distant from the gaze point for 
the immediately preceding image by one degree or more in terms of angle of convergence (step SI 09). The position 
data of the model thus moved (with notification that a limitation has been imposed on the movement of the gaze point. 
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if so) is sent to and stored in the model data manage means 19. 

The model data manage means 1 9 then sends data on the distance over which the model has moved to the model 
enlarge/reduce means 15. Upon receipt of this data, the model enlarge/reduce means changes the size of the model 
according to that distance (step S111). If, for example, the gaze point P is made coincident with the display screen, 

5 the size of the model is changed in proportion to the distance between the viewpoint center and the gaze point so that 
a relation is established such that (size of model prior to movement) : (size of model after movement) = (Z-axis coor- 
dinate Zd of gaze point prior to movement) : (Z-axis coordinate of gaze point after movement). This makes the image 
size of the model on viewer's retina remain unchanged before and after the movement; thus, the viewer will not notice 
the movement of the model in the direction of depth. Thus, even when an attempt is made to rotate the modei for 

10 viewing from different angles, the viewer will not have a feeling of strangeness such that the model sudden iy comes 
up or goes away at the time of rewriting the image. This is therefore useful in reducing the viewer's eye fatigue. The 
data on the size of the model thus changed is sent to the model data manage means 19 and stored in it. 

Further, it is not necessary to always change the size of the model so that the apparent model size remain un- 
changed. 

15 The model manage means 1 9 then sends the model position data and the model size data to the depth distance 

reduce means 16. Here, the perceptual distance of depth is reduced by changing the interocular distance. The model 
manage means 1 9 makes a decision from the graph of FIG. 1 0 as to whether or not the entire model has been placed 
within the easiness-of-stereoscopy region, on the basis of the interocular distance set as the initial value. 

When part of the model is outside the easiness-of-stereoscopy region, the depth distance reduce means 1 6 corrects 

20 the initial value of interocular distance so that it becomes reduced (step S113). By reducing the interocular distance 
from DO to D1 , the offset amount of the left and right images is reduced from do to dl as shown in FIGS. 11 A and 1 1 B. 
Whether the stereoscopy is easy depends on whether that offset falls within a predetermined range. Thus, reducing 
the interocular distance appropriately will allow the entire model to fit into the easiness-of-stereoscopy region. 

FIG. 10 is a graph showing the criterion for correcting the interocular distance with the distance between the 

25 viewpoint center and the display screen held constant. The axis of ordinate indicates the interocular distance, and the 
axis of abscissa indicates the depth distance between the model and the viewpoint center How to create this graph 
wiil be discussed below. 

First, the depth distances AnI and Afl between both ends of the easiness-of-stereoscopy region and the viewpoint 
center O corresponding to the distance between the viewpoint center O and the display screen 1 a (60 cm in the present 
30 embodiment) are sought. Thereby, the depth distance between each end of the easiness-of-stereoscopy region and 
the viewpoint center O for the original interocular distance DO set as the initial value is obtained. Thus, points L and M 
in FIG. 10 are determined. 

Next, assuming that, in FIG. 12A, the model element E locates at the front end of the easiness-of-stereoscopy 

region on the viewer side, the positions of the left image IL and the right image IR for the element E on the display 
35 screen la are sought to calculate the amount dO that the images iR and IL are offset with respect to each other This 

offset dO corresponds to the maximum allowable offset for images IL and IR indicating an element that is located at 

the front end of the easiness-of-stereoscopy region on the viewer side 

Next, as shown in FIG. 12A, considers a modei eiement E' that is located at a point that is at a distance of An from 

the viewpoint center O, that is, the point is on the viewer's side and outside the readiness-of-stereoscopy region. 
40 Straight iines that extend from the images IL and IR and intersect the element E' are then drawn to seek the intersections 

of the straight lines and the X axis. The distance D between those intersections is the maximum value of the interocular 

distance which provides easy stereoscopy for the model element E' that is located at a distance of An from the viewpoint 

center O. 

When the model element E' is very close to the viewpoint center, for example, when the depth distance between 
45 the viewpoint center and the element E' is less than 1/1 0 of the distance between the viewpoint center and the display 
screen, the interocular distance becomes very small, so that the three-dimensional effect is lost due to little parallax. 
When the depth distance between the model element E' and the viewpoint O is less than 1/10 of the distance Ai, the 
interocular distance is set to the value when the depth distance between E' and O is 1/10 of Ai. When a model is very 
ciose to the viewpoint center, its apparent size increases, so that a portion of the model which is pushed out of the field 
50 of view increases and the shape recognition as a whole becomes difficult. Thus, even if images which are offset more 
than the maximum allowable offset amount are dispiayed. the viewer wili not care about such an image. 

A curve KL in the graph of FIG. 10 can be defined by changing the depth distance An from the viewpoint center 
O, seeking the maximum value of the interocular distance D that corresponds to the changed distance An, and piotting 
the distance An and the interocular distance D for An on the graph. Using this curve, the interocuiar distance can be 
55 determined which allows the stereoscopy of an element that is located in front of the easiness-of-stereoscopy region. 

Further, by referring to FIG. 12B, a curve MM can be determined in the same way as the above approach, which 
indicates the criterion for correcting the interocular distance for facilitating the stereoscopy of an element that is located 
beyond the back end of the easiness-of-stereoscopy region. 
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On the basis of the graph of FIG. 10 thus obtained, the depth distance reduce means 16 corrects the initial value 
of the interocular distance so that it becomes small. As shown in FIG. 1 0, suppose that model elements are distributed 
over a range in depth direction centered at the gaze point P and indicated by arrows R In this case, if, when the depth 
distance between the viewpoint center O and an element that is situated at the most front end (on the viewer side) of 
5 that range is An, the interocular distance is set to D according to the curve KL, then all the model elements can be 
easily viewed stereoscopically 

Suppose, for example, the case where only one element is situated far behind the other elements. In such a case, 
it is not necessarily required that all the elements be placed in the easiness-of-stereoscopy region. That is, it is allowed 
to place at least main model elements in the easiness-of-stereoscopy region. 
10 When the entire model is placed in the easiness-of-stereoscopy region at the termination of step S111 , the depth 

distance reduce means 16 keeps the Interocular distance as it was initially set. 

The interocular distance set in the above manner is sent to the model manage means 1 9. 

For processing based on the whole gaze range (as opposed to the gaze point P), reference can be made to the 
position of the front end of the gaze range instead of the model element situated nearest to the viewer and to the 
15 position of the back end of the gaze range instead of the farthest model element, thereby allowing the movement and 
enlargement of models and the change of the interocular distance in the same way as above. 

The model data manage means 1 9 sends the model position data, model size data, and interocular distance data 
to the tone change means 1 7, which, in turn, makes a decision as to whether or not the effect of seeing objects through 
air is required with an image being displayed (step S1 14). 
20 It is when many model elements are situated behind the gaze point that the effect of seeing objects through air is 

needed. In such a case, even if back elements differ from one another in depth coordinate (i.e., Z-axis coordinate), the 
amount that, when they are actually displayed on the screen, their associated right and left images are offset with 
respect to each other is not so great; thus, it is difficult for the viewer to recognize differences in depth among the 
elements. 

25 As another example, the effect of seeing objects through air Is needed when the interocular distance set by the 

depth distance reduce means 16 is small. The reason is that, when the interocular distance is set small, the amount 
that right-eye and left-eye images displayed on the display 1 are offset with respect to each other becomes smaller 
than with the case where the interocular distance is set large and hence It becomes difficult for the viewer to recognize 
the difference in depth among elements. 

30 When the decision is that the effect of seeing objects through air is needed, the tone change means 17 changes 

the tone of the model so that the further away each model Is from the viewpoint center O, the closer its tone gets to 
the background tone (step 8115). When the decision is that the effect of seeing objects through air is not needed, the 
tone change means 17 keeps the model tone as it stands. 

The effect of seeing objects through air may be furnished all the time or otherwise. 

35 The tone of the model element thus changed (or kept) Is sent to the model data manage means 19. 

At the termination of the processing through step S115, the model data manage means 19 will have held the depth 
position in the stereoscopic coordinate system of the model that is to be displayed on the display screen, the size 
(including the shape) and tone of the model, and the interocular distance serving as a basis for calculation of the display 
positions of images of that model (right- and left -eye Images) that are to be displayed on the display screen. 

40 The model data manage means 19 then sends such data to the image data create means 20. In addition, the 

control means 1 0 reads data relating to graphical user interface components to be displayed on the display from the 
GUI data manage means 21 and passes them to the image data create means 20. The image data create means 
places the graphical user interface components on a plane parallel to the XY plane having the same Z-axis coordinate 
as the gaze point after being moved in the stereoscopic coordinate system by the model move means 14 or in the 

45 vicinity of that plane (step S1 16). 

Based on the data, the image data create means 20 determines the display positions and display sizes of rlght- 
and left-eye images of each model element on the display screen la and moreover determines the positions in the 
stereoscopic coordinate system of the graphical user interface components to be displayed on the display (step 8117). 
Hereinafter, reference will be made to FIGS. 13A and 13B to describe the operation of the image data create 

so means 20. First, description will be given of the case where an element El constituting a model is located in front of 
the display screen la in the stereoscopic coordinate system. 

The line LL connecting the left eye 3a and the center GE1 of the element El Is first set up and then the intersection 
CL1 of the line LL and the display screen la is obtained. This intersection CL1 is determined as the center position 
(display position) of the left-eye image I LI of the element El displayed on the display screen la. The size of the left- 

ss eye image I LI is determined so that the ratio of the distance between the left eye 3a and the element E1 to the distance 
between the left eye 3a and the intersect GR1 becomes equal to the ratio of the size of the element El to the size of 
the left-eye image ILL On the basis of this determination, image data for the left eye is created. For the right-eye image 
IR1 as well, its center position CR1 and size are determined in the same way and then image data for the right eye is 
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created. 

FIG. 13B shows the case where a model element E2 is located behind the display screen la in the stereoscopic 
coordinate system. In this case as well, the center position and size of each of the images for the left and right eyes 
are determined in the same way as described in conjunction with FIG. 1 3A. 

5 The image data create means 20 also performs the same processing for graphical user Interface components to 

determine the display position and size of each of the left- and right-eye images for each GUI component. 

When the elements El and E2 are displayed simultaneously and, In step S115, the model's tone is changed by 
the tone change means 17 in such a way that the farther away from the viewpoint center, the closer the model tone 
gets to the background tone, the image data create means 20 creates image data such that the tone of left and right 

10 Images IL2 and IR2 for the element E2 is closer to the background tone than that of left and right images IL1 and IR2 
for the element El. 

Further, when the elements E1 and E2 are displayed simultaneously and the left and/or right Image for the element 
E2 overlaps with the left and/or right Image for the element El, the Image create means 20 adopts, on preferential 
basis, the image data for the element El that is present In front of the screen and deletes Image data which, of Image 
15 data for the element E2 that is present behind the screen, is associated with that portion of the element E2 which 
overlaps the image of the element El . 

The image data create means 20 then sends the left-eye image data and the right-eye image data to the display 
unit 1, which, in turn, displays images based on the image data (step S118). 

Next, the control means 10 inquires of the viewer as to whether he or she will continue to use the stereoscopic 
20 system (step SI 03). When, for example, the viewer wants to display the model with its orientation changed, the process 
In step SI 04 and the subsequent processes are carried out. 

As described above, according to the present embodiment, an Image Indicating a model can be automatically 
displayed under the optimum stereoscopic conditions. For this reason, a feeling of strangeness the viewer has for the 
displayed images and viewer's eye fatigue can be eased. 
25 in the stereoscopic system according to this embodiment: to further ease the viewer's eye fatigue, the following 

approaches are taken. That Is, the binocular fusing type stereoscopic system can Impose constant strain on viewer's 
eyes In comparison with the normal two-dimensional display system. For this reason, to further ease the strain on the 
viewer, the system is provided, as shown in FIG. 1 , with the offset adjust means 22 that adjusts the amount that right 
and left images are offset with respect to each other 
30 The offset adjust means 22 monitors image display time and, when a fixed period of time lapses after the start of 

display. Issues an offset adjust instruction to the image data create means 20 via the model data manage means 1 9. 

The stereoscopic effect depends on the amount that the right and left Images are offset with respect to each other 
(the distance between GL1 and GR1 in FIG. 1 3A). The smaller the offset, the smaller the stereoscopic effect becomes 
and vice versa. When the offset is too great, it becomes impossible for viewers to fuse right and left images Into one 
35 three-dimensional Image, that is, stereoscopy itself becomes impossible, 

in the present embodiment, in response to that Instruction the data create means 20 varies the offset amount as 
shown In FIGS. 18A and 18B with the distance between CL1 and CR1 in FIG, ISA and the distance between GL2 and 
CR2 in FIG, 13B as critical offset amounts. 

That Is, as shown in FIG, 18A, until a given period of time, t1 (for example, twenty minutes), lapses after the start 
40 of display, the offset amount corresponds to the critical offset amount and, after an elapse of t1 , the data create means 
20 varies slowly the offset amount with a given period to diminish the three-dimensional effect. 

Such processing prevents viewer's watching images with an enhanced effect of three dimensions for a long time, 
allowing their eye fatigue to be eased. 

Such processing as shown in FIG. 18B can also be performed. In this example, at the start of Image display, the 
45 offset amount is set smaller than the critical value so as to diminish the three-dimensional effect, and the offset amount 
is then varied with time to approach the critical value. 

That is, the stereoscopy Is realized by viewer fusing right and left images into one image In the brain. It is therefore 
considered that since, at the start of view, the viewer is not accustomed to fusing, he or she will not be able to perform 
fusing well or great strain will be imposed on the eyes. This problem will be solved by performing the processing shown 
so in FIG, IBB. 

in the present embodiment, in determining the stereoscopic conditions, the entire model is moved on the coordinate 
system for the stereoscopic system (steps S109 and S110). As long as the movement alone falls to place the entire 
model in the easiness-of-stereoscopy region, the interocular distance is corrected to thereby reduce the space in the 
direction of depth (step S1 15), which provides stereoscopy with a sufficient effect of three dimensions in comparison 
55 with simply reducing the space in the direction of depth. 

Hereinafter, reference will be made to FIGS. 14A, 14B, and 14C and FIGS. 15A and 15B to describe problems 
that arise from the preferential correction of interocular distance. 

Suppose that, as shown in FIG. 14A, model elements El, E2 and E3 are placed on the stereoscopic coordinate 



13 



EP 0 817 123 A1 



system. When the interocular distance is the initial value DO, left images IL1, IL2 and IL3 and right images IR1, IR2 
and IRS of the respective elements El, E2 and E3 are displayed on the display screen la. Suppose here that the 
interocular distance is reduced from DO to D as shown in FIG. 1 4B. Then, left images IL1 ', IL2' and IL3' and right images 
IR1', IR2'and IR3'of the respective the elements El, E2 and E3 are displayed on the display screen 1a. An advantage 
with the way to change the interocular distance is that each image prior to change of the interocular distance and the 
corresponding image after change of the interocular distance are equal to each other in size. 

The interocular distance correction is only performed in image processing; the viewer's actual interocular distance 
is not changed. For this reason, the viewer will perceive the elements El, E3, and E3 as if they were placed in the 
positions El ', E2' and E3'. In this case, the viewer will perceive the elements El , E2 and E3 as if the interval between 
each element were reduced considerably in the direction of depth. 

The present embodiment solves the above problems by performing the movement of the entire model on prefer- 
ential basis. That Is, as shown in FIG. 15B, the present embodiment moves the elements El , E2 and E3 in the direction 
of depth on the stereoscopic coordinate system prior to the interocular distance correction. Thus, the positional relation 
among the elements E1 , E2 and E3 that the viewer perceives will not vary as shown in FIGS. 15Aand15B. However, 
the movement of the entire model varies the size of the left and right images displayed on the display screen la. To 
compensate for variations in size, the size of each element is corrected according the distance over which it has been 
moved (step S1 1 1 ). In the present embodiment, the interocular correction is made only on that portion of a model for 
which only the movement of the entire model is insufficient to fit the model into the easiness-of-stereoscopy region. 
Thus, the loss of the sensation of depth for each model element can be reduced to a minimum. 

In the right and left image fusing type stereoscopic system, in displaying a model that is situated at the edge of 
the screen before the screen, a certain point of the model may be displayed in only one of right and left images. In 
such a case, the viewer will perceive that the image of the model that is to be originally present before the screen is 
hidden by the screen that is to be present behind the image; thus, a discrepancy occurs in a forward and backward 
position relation. 

There will be two solutions for that problem; the first Is to set up all the display objects (model) behind the screen 
and the second is not to make the viewer conscious of the screen edges definitely With the first method, the protrusion 
of displayed images is small, failing to obtain pictures of full power For this reason, the system of this embodiment 
implements the second method by the screen edge processing section 23 shown in FIG. 1 . 

The screen edge processing section 23 receives model position data, model size data, and interocular distance 
data from the model data manage means 1 9. On the basis of these data, the screen edge processing section 23 detects 
whether the model Is situated at the screen edge and determines whether the edge blurring or shading-off process Is 
to be performed. In addition, the range of blurring of shading-off of the edges of right and left images is determined. 

Based on these determinations, the screen edge processing section 23 changes the lightness of the model step- 
wise so that the closer to the edge, the darker, or changes the tone of the model so that the closer to the edge, the 
closer to the background tone. 

This Image data is sent to the image data create means 20 together with other data via the model data manage 
means 19 and processed in step S1 18. 

Such processing makes the viewer somehow feel the screen edge Invisible. Even in the presence of such a dis- 
crepancy as described above, therefore, the viewer will become unconscious of it. 

Next, a second embodiment of the present invention will be described with reference to FIGS. 16, 17A and 17B. 
As shown in FIG. 16, the second embodiment Is distinct from the first embodiment only in that a visual-line/gaze range 
detect means 30 replaces the gaze range determination means 13 in the first embodiment and the stereoscopic con- 
dition calculation flow is changed accordingly. Thus, like reference numerals are used to denote corresponding com- 
ponents to those in the first embodiment and detailed descriptions thereof are omitted. 

As shown in FIG. 16, the stereoscopic system Is equipped with the visual-llne/gaze range detect means 30 which 
detects actual visual lines and gaze point on displayed Images. The means consists of a binocular gaze analyzer 
manufactured by Takel company as an example. This means Is connected to the model data manage means 19 to 
send to it in real time information as to which area of the display screen la the viewer's eyes are fixed on. As the visual- 
line/gaze range detect means 30, use may be made of existing binocular gaze analyzers having the same function as 
the one manufactured by Takel company. 

The operation of the embodiment thus arranged will be described with reference to FIGS. 1 7A and 1 7B. 

As in the first embodiment, steps S100 to S104 are carried out as shown in FIG. 17A. 

Next, the model data manage means 1 9 receives light-of-sight and gaze point position data from the visual-line/ 
gaze point detect means 30 (step S200) and then makes a decision as to whether or not the viewer watches the screen 
from the detected direction of visual lines (step S201). When the viewer is not watching the display screen la, the 
center 1 b of the screen is taken as the gaze point (step S202). When the viewer is watching the screen, on the other 
hand, the point on the screen at which the viewer's visual lines are fixed for the longest time is taken as the gaze point 
(step S203). 
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In this case, the gaze range in the depth direction can be determined as a range of standard deviation from gaze 
points, coordinate of Z-axis in the model. 

As in the first embodiment, the processes in steps S1 10 to S1 15 are carried out next, so that the model data 
manage means 1 9 obtains data on the shape, size, placement and tone of the model in the stereoscopic coordinate 
5 system. The data are sent to the image data create means 20 to produce image data for right and left eyes (step S11 7). 
The display unit 1 displays images for the right and left eyes on the display screen la (step S118). 

The control means 1 0 next inquires of the viewer as to whether he or she continues to use the stereoscopic system 
(step SI 03). When the viewer continues to use the system, the procedure is repeated starting with step SI 04. 

As described above, according to the second embodiment, the visual-line/gaze range detect means 30 is provided 
10 which detects viewer's actual visual lines and gaze point, allowing the gaze range to be determined more accurately. 
For this reason, more proper stereoscopic conditions can be calculated. 

In the first and second embodiment described above, the model move means 14 and the depth distance reduce 
means 16 function to put the gaze range or the entire model Into the easiness-of-stereoscopy region shown in FIG. 9. 
This is not restrictive. They may function to put the gaze range or the entire model into the possibility-of-stereoscopy 
15 region. 

A third embodiment of the present invention will be described with reference to FIGS. 1 9, 20A, and 20B. 

The first and second embodiments are arranged to move a model and enlarge or reduce it to cause an eye gaze 
range for that model to be positioned in the possibility-of-stereoscopy region, thereby allowing stereoscopy to be fa- 
cilitated. In this embodiment, the same processing is performed by a perspective transformation method determined 
20 by a perspective transformation determining means 32 and stored in a perspective transformation storage means 33. 

Hereinafter, the operation of the system will be described with reference to FIG. 20A and 20B. Like reference 
numerals are used to denote corresponding components to those in the first embodiment and descriptions thereof are 
omitted. 

The first and second embodiments take the processes of changing the position of a model, enlarging it, and chang- 
25 ing the interocular distance in order to produce stereoscopic images. From a different point of view, the processes can 
be organ ized as a perspective transformation method for projecting a three-dimensional model onto a two-dimensional 

image. 

Let the interocular distance be 2e, the viewpoint center be 0(0, 0, 0) and the distance between the viewpoint center 
and the display screen be D. According to the perspective transformation, a point P(xp, yp, zp) in three-dimensional 
30 space is then projected onto: 
Left-eye image: 

(D-xp/Zp+e-(D/zp-1), D-yp/zp) 

35 

Right-eye image: 

(D-xp/Zp-e-(D/zp-1), D-yp/zp) 

Thus, the use of the perspective transformation allows the same processing to be performed without moving and 
enlarging/reducing a three-dimensional model as in the first embodiment. 

That is, in this system, a gaze range is obtained in step S301 of FIG. 20A in the same way as described above 
and, in subsequent step S302, an enlargement ratio m (corresponding to a perspective transformation method) for 
45 space centered at the viewpoint center Is determined from the original position of the model and the distance required 
to move the model into the gaze range. 

By enlarging or reducing the whole space in which the model is situated by the space enlargement ratio m, the 
size of the model is increased or decreased by a factor of m and a point P in the model is shifted to a point P' (m xp, 
myp, mzp). 

50 In subsequent Step 8303, a decision is made as to whether each point in the model after perspective transformation 

is in the easiness-of-stereoscopy region. That is, a point N' to which a representative point P that is nearest to the 
viewpoint center has been shifted and a point M' to which a representative point M that is farthest from the viewpoint 
center has been shifted are calculated and the above decision is made for those points. 

When the decision is that the points are not in the easiness-of-stereoscopy region, the interocular distance is 
55 adjusted In the same way as described in conjunction with FIG. 10 to compress the space in the direction of depth so 
that all the points become positioned in the easiness-of-stereoscopy region. Assuming here that the reduction ratio for 
the interocular distance is t, that point P in the three-dimensional space is, in the ultimate analysis, projected onto: 
Left-eye image: 



15 



EP 0 817 123 A1 



(D xp/Zp+e • t • (D/(m • zp)-1 , D-yp/zp) 

Right image; 

5 

(D • p/Zp-e • t • (D/(m • zp)-1), D-yp/zp) 

TPius, tlie perspective transformation metliod can be defined by tine model enlargement ratio m in relation to the 
10 viewpoint center, the original interocular distance 2e, the distance D betw/een the viewpoint center and the display 
screen, and the Interocular distance reduction ratio t obtained from FIG. 10 (step S305). 

Those data (perspective transformation method) are preserved in the perspective transformation storage means 
33 and employed by the image data create means 20 for subsequent Image data creation (steps S116, S117 and 
S118). In this manner, image data that allows stereoscopy to be facilitated can be created as in the first and second 
15 embodiments. 

In this example, the perspective transformation method Is stored In the perspective transformation storage means 
33. This Is not restrictive. Astorage medium, such as a DVD, may be used to store a perspective transformation method 
for each scene obtained In advance by the system of the present embodiment together with the model position within 
the field of view seen from viewpoints. 
20 That Is, by setting In the storage medium the Interocular distance, the distance between the viewpoint center and 

the display screen, the model data prior to processing, the position of the model seen from the viewpoint center, the 
model enlargement ratio, m, and the Interocular distance reduction ratio, t, for each scene, the Image data create 
means 20 can produce images that are easy to view stereoscoplcally without the use of the stereoscopy facilitating 
condition calculating section. 

25 A fourth embodiment of the present invention will be described below with reference to FIGS. 21 , 22A and 22B. 

In these figures, like reference numerals are used to denote corresponding components to those in the first embodiment 
and descriptions thereof are omitted. 

The first, second and third embodiments are each directed to a system which displays a three-dimensional model 
given by information in the form of shape, size, position of placement, and so on so that it can easily be viewed ster- 
30 eoscoplcally. On the other hand, the fourth embodiment Is directed to a system which displays actually shot pictures, 
such as motion pictures and photographs, in a form that they are easy to view stereoscoplcally 

In such a case, whether the stereoscopy is easy depends on the positional relationship between gaze ranges In 
left and right Images. For this reason, the system of the fourth embodiment Is equipped, as shown In FIG. 21 , with an 
Image conversion determination means 34 which determines a conversion method for converting Image data to image 
35 data that are easy to view stereoscoplcally and an Image operations means 35 for obtaining converted image data in 
accordance with the image conversion method determined. 

The process flow in this system will be described with reference to FIG. 22A and 22B, In this figure, like reference 
numerals are used to denote corresponding steps to those in the first embodiment and descriptions thereof are omitted. 
The system uses two parallactic images, such as an Image shot with a right-eye camera and an image shot with 
40 a left-eye camera. The Image data are sent to the eye gaze range determination means 13 to determine the gaze 
range using either of the Images (step S401). 

To determine the gaze area, a method Is taken which obtains the center of gravity of pixels weighted by tones and 
the standard deviation a of distance from the center of gravity and determines a circle with a radius of 2a from the 
center of gravity as the gaze area. Assuming the coordinates of each pixel to be (xi, yl), the center of gravity (xG, yG) 
45 and the standard deviation a are represented by equations (7) and (8) 
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An alternative method is to detect corresponding image areas in tlie riglitand left images, seel< tliree-dimensionai 
coordinates from tlie correspondence between tlie image areas, and determine tfie gaze range in tiie direction of deptii 
as witli tine model in the first, second and third embodiments. 

To find corresponding points in the image areas, there is a method which emphasizes the edges of both the images 
and seeks points having a high level of correlation (see "multi-stage stereoscopic method that prioritizes correspond- 
ence of high reiiabiiity" by Yamaguchi et ai, The Institute of Electronics, Information and Communication Engineers 
Paper D-11, Voi.J74-D-11, No.7). 

Next, in step S402, a search is made under the following two conditions for a corresponding point for which, of 
corresponding points in other areas than the bacl^ground within the determined gaze area, the amount that the right 
and left images are offset with respect to each other is largest. 

That is, when the coordinate origin is fallen on the left side of the display screen, the X-axis coordinate of one point 
(corresponding point) in the left image is fallen as XL and the X-axis coordinate of that point in the right image is fallen 
as XR. As the first condition, a search is made of points for which XL >XR for a point S for which XL-XR is at maximum. 

At the point S, the left image is larger in X-axis coordinate than the right image. That Is, the point S is recognized 
as a point which, like El shown in FIG. 1 3A, is situated before the screen. Of such points, the point S for which the 
amount that the right and left images are offset with respect to each other Is largest Is recognized as being situated 
nearest to the viewer within the gaze range. 

Next, as the second condition, a search is made of points for which Xr > XI for a point T for which Xr - XI is at 
maximum. The point T Is recognized as being situated behind the screen within the gaze range as shown In FIG. 13B 
at E2. Of such points, the point T Is recognized as being situated farthest from the viewer within the gaze range because 
the amount that the right and left images are offset with respect to each other is largest for T. 

Assume here that the position of the point S in the left image is (Xpl, Yp) and the position in the right image Is (Xpr, 
Yp), and for point S the amount that the left Image and the right image are offset with respect to each other. I.e., Xpl - 
Xpr, is dXp. Assume that, for the point T, the position In the left image is (Xql, Yq) and the position in the right image 
is (Xqr, Yq), and the amount that the left image and the right image are offset with respect to each other, i.e., Xqr - Xqi, 
is dXq. 

In the following, consider, as an example, the case where dXp > dXq, that is, the case where the offset amount 
for point S is larger than that for point T. A decision is made as to whether the point S lies within the easiness-of- 
stereoscopic range. In this embodiment, this decision is made in step S404 using an visual angle. 

In this case, the maximum image offset within the gaze range corresponds to dXp for point S. A decision is made 
as to whether the offset exceeds two degrees in terms of visual angle. The reason is that, as described above, the 
easiness of stereoscopy is lost when the amount that the left and right images are offset with respect to each other is 
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two degrees or more in terms of visual angle. 

When, in step S403, the offset amount Is less than two degrees in terms of visual angle, it is not particularly required 
to change image data. Thus, this offset amount in terms of visual angle is used as a set value. When the offset amount 
is two degrees or more, however, the set value is taken to be just two degrees in step S404. 
5 In this case, it is also required to take a variation in the offset into consideration. The reason is that, as described 

above, the stereoscopy becomes difficult when a variation in the offset becomes one degree or more in terms of angle 
of convergence. 

In step S405, therefore, the set value is determined such that the variation is less than one degree when the 
variation Is one degree or more. In this case, the set value for the visual angle is taken to be not just two degrees but 
10 a value such that the variation is maintained at one degree (step S406). 

The visual angle (set value) is determined from the distance D between the viewpoint center and the display screen. 
That is, the approximate distance (L) on the screen for a visual angle of two degrees will be 

,5 L = 2 X D Xtan(7i/180) 

= 71 X D/90 
= 0.035 X D 

20 

In subsequent step S407, the space enlargement ratio m is determined using this set value. Here, assuming that 
the interocular distance is 2e and the set value for visual angle is two degrees, the space enlargement ratio m is defined 
by 

25 

m = (dxp+2 X e)/(L+2 X e) 

When the Image Is enlarged or reduced by a factor of the space enlargement ratio, the entire gaze range may not 
fall within the easlness-of-stereoscopy region. In this case. In step S408, the reduction ratio t In the horizontal direction 
30 is obtained using the offset amount dXq for the point S. 
That is, the reduction ratio t will be: 
When (dxq+2 X e)/m-2e < 0.035D, 



When (dxq+2xe)/m-2e > 0.035D, 

t = 0.035XmXD/(dxq+2e-2em) 

When the space enlargement ratio and the horizontal reduction ratlot have been obtained In this manner, an image 
operations method is determined in step S409. 

According to the image operations method of the embodiment, any corresponding points (xal, y), (xar, y) on the 
45 left and right images are transformed into: 

left image: 

(Xal+Xar)/2 + eXtX((Xal-Xar+2 X e)/(2 Xm X e)-1) 

right image: 

(Xal+Xar)/2 - extx ((Xal-Xar+2 X e)/(2 xm X e)-1 ) 

The image operations means 35 converts image data with each scene, and the image data create means 20 
produces stereoscopic images. Thus, as with the first, second and third embodiments, such a system can produce 
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image data that allow easy stereoscopy even for actually shot images. 

In this example, image conversion is performed with each scene by the image operations means 35. By storing 
on a storage medium, such as a DVD, image data that have been subjected to image conversion in advance by the 
stereoscopic condition calculation section of this system , the image data create means 20 can produce images that 
provide easy stereoscopy without the use of the stereoscopic condition calculation section. 

The system of the fourth embodiment is effective in producing stereoscopic images easy to view on the basis of 
pictures shot by two cameras from different viewpoints. 

Further, the system is also applicable to a system which produces a time difference from a moving image for 
stereoscopy. Moreover, the present invention is also applicabie to binocuiar stereoscopic systems other than the time- 
muitiplexing stereoscopic system. 

As described above, according to the present invention, images representing a model placed in three-dimensional 
space can be automatically displayed on a display unit under optimum stereoscopic conditions. For this reason. It Is 
possible to diminish the viewer's feeling of strangeness for images and eye fatigue. 



1 . A stereoscopic display system for displaying a model stereoscopically using a pair of images seen from two different 
viewpoints and adapted for the right eye and the left eye, characterized by comprising: 

gaze range determining means(13) for determining a range to which a viewer is likely to fix his or her eyes 
with respect to the model; 

model move means(14) for moving the model in a direction perpendicular to a display screen so that the gaze 
range determined by the gaze range determining means(13) comes in a region in which stereoscopy is easy 
by changing data of the model; 

model enlargement/reduction means(1 5) for enlarging/reducing a size of the model moved by the model move 
means(14) according to a distance over which the model is moved by changing data of the model changed 
by the model move means(14); and 

Image data creating means(20) for creating Image data to be displayed on the display according to data of 
model which has been moved by the model move means (1 4) and whose size has been changed by the model 
enlargement/reduction means(15). 

2. The system according to claim 1 , characterized by further comprising depth distance reducing means(1 6) for setting 
a scale in the direction of depth seen from the viewer so that at least the major part of a model moved by the model 
move means(14) comes in the region in which stereoscopy is easy 

3. The system according to claim 1 , characterized by further comprising a visual-line/gaze range detecting means 
(30) for detecting viewer's visual line and gaze range with respect to an image actually displayed, wherein the 
model move means(14) moves the model in a direction perpendicular to the display according to a gaze range 
detected by the visual-line/gaze range detecting means(30). 

4. The system according to claim 1, characterized in that the model move means(14) includes a means for, when a 
change occurs in the placement condition of the model seen from the viewpoint and the model move means moves 
the model according to the change, preventing an amount of moving of the model within the gaze range from 
exceeding about one degree In terms of angle of convergence for a predetermined period of time after the change 
in the placement condition occurs. 

5. A stereoscopic display system for displaying a model stereoscopically using a pair of Images seen from two different 
viewpoints, characterized by comprising: 

gaze range determining means(13) for determining a range to which a viewer is likeiy to fix his or her eyes 
with respect to the model; 

a perspective transformation determining means(32) for determining a perspective transformation method of 
the model data to the pair of images for the left eye and the right eye according to an output of the gaze range 
determining means (13); and 

an image data creating means(20) for creating the image data to be displayed on the display according to a 
perspective transformation method determined by the perspective transformation determining means(32). 
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6. The system according to claim 1 or 5, cinaracterized in tinat the gaze range determining means(1 3) has a means 
tor estimating a gaze range for both eyes by taking into consideration all or part of placement condition, tone, 
brightness, saturation, size, complexity and moving speed of the model seen from a viewpoint of a viewer. 

5 7. The system according to claim 1 or 5, characterized by further comprising out-of-field data removing means(12) 
for removing data of a model existing out of field of view. 

8. A stereoscopic display system according to claim 7, characterized by further comprising a visual-line/gaze range 
detecting means(30) for detecting viewer's visual line and gaze range with respect to an image actually displayed, 

10 wherein the out-of-fieid removing means(30) calculates a field of view on the basis of a visual line detected by the 

visual-line/gaze range detecting means(12) and removes data existing out of the field of view. 

9. The system according to claim 1 or 5, characterized by further comprising a tone changing means(1 7) for changing 
atone of a displayed model so that the further away from the viewpoint, the closer the tone becomes a background 

is tone. 

10. The system according to claim 1 or 5, characterized in that the image data creating means(20) has a function of 
determining stereoscopic condition of graphic user interface items to be displayed on the display screen. 

20 1 1 . A stereoscopic display system for displaying a pair of images seen from two different viewpoints stereoscopically 
characterized by comprising: 

gaze area determining means(1 3) for dctormining a range to which a viewer is likely to pay attention; 
an image conversion determining means(34) for dctormining a method for image conversion so that the gaze 
25 area becomes a region in which stereoscopy is easy according to an output of the gaze area determining 

means(13); and 

an image data creating means(20) for creating the image data to be displayed on the display according to an 
output of the Image conversion determining means{34). 

30 12. A stereoscopic method for displaying a pair of picture images seen from two different viewpoints with respect to 
a model on a display screen stereoscopically, characterized by comprising: 

gaze range determining step(S106) for determining a range to which a viewer is likely to pay attention with 
respect to a model; 

35 a model moving step (S110) for moving the model in a direction perpendicular to a display screen so that the 

gaze range determined by the gaze range determining step(S106) comes in a region in which stereoscopy is 

easy by changing data of the model; 

a model eniarging/reducing step (S111 ) for enlarging/reducing a size of the model moved by the model moving 
step (S110) according to a distance over which the model is moved by changing data of the model changed 
40 by the model moving step (S1 1 0); and 

a data creating step (S1 1 7) for creating image data to be displayed on the display according to data of model 
which has been moved by the model moving step (S110) means and whose size has been changed by the 
model enlargement/reduction step(SIII). 

45 13. The method according to claim 12, characterized by further comprising a depth distance reducing step(S113) for 
setting a scale In the direction of depth seen from the viewer so that at least the major part of a model moved by 
the model moving step (S110) comes in the region in which stereoscopy is easy. 

14. The method according to claim 12, characterized by further comprising a visual-line/gaze range detecting step 
50 (S201 , 8203) for detecting viewer's visual line and gaze range with respect to an image actually displayed, wherein 

the model moving step (8110) moves the model in a direction perpendicular to the display according to a gaze 
point detected by the visual-line/gaze range detecting step(8203). 

15. The method according to claim 12, characterized in that the model moving step (S110) includes a means for, when 
55 a change occurs in the placement condition of the model seen from the viewpoint and the model moving step 

(8110) moves the model according to the change, preventing an amount of moving of the model within the gaze 
range from exceeding about one degree in terms of angle of convergence for a predetermined period of time after 
the change in the placement condition occurs. 
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16. A stereoscopic display metliod for displaying a model stereoscopically using a pair of images seen from two dif- 
ferent viewpoints, comprising: 

a gaze range determining step(S301 ) for determining a range to which a viewer is likely to fix his or her eyes 
5 with respect to the model; 

a perspective transformation determining step(S302) for determining a perspective transformation method of 
the model data to the pair of images for the left eye and the right eye according to an output of the gaze range 
determining step(S301); and 

an Image data creating step(S117) for creating the Image data to be displayed on the display according to a 
TO perspective transformation method determined by the perspective transformation determining step(S302). 

17. The method according to claim 12 or 16, characterized in that the gaze range determining step(S106, S301) has 
a step for estimating a gaze range for both eyes by taking into consideration all or part of placement condition, 
tone, brightness, saturation, size, complexity and moving speed of the model seen from a viewpoint of a viewer 

1S 

18. The method according to claim 1 2 or 1 6, characterized by further comprising out-of -field data removing step(S200) 
for removing data of a model existing out of field of view. 

19. The method according to claim 18, characterized by further comprising a visual-line/gaze range detecting step 
20 (S201 ,S202) for detecting viewer's visual line and gaze point with respect to an image actually displayed, wherein 

the out-of-field removing step(S200) calculates a field of view on the basis of a visual line detected by the visual- 
line/gaze range detecting step(S201 , S202) and removes data existing out of the field of view. 

20. The method according to claim 1 2 or 1 6, characterized by further comprising atone changing step(S1 1 5) for chang- 
25 ing a tone of a displayed model so that the further away from the viewpoint, the closer the tone becomes a back- 
ground tone. 

21 . The method according to claim 1 2 or 1 6, characterized In that the Image data creating step(S1 1 7) has a step(S11 6) 
of determining stereoscopic condition of graphic user interface Items to be displayed on the display screen. 

22. A stereoscopic display method for displaying a model stereoscopically using a pair of Images seen from two dif- 
ferent viewpoints, characterized by comprising: 

gaze area determining step(S401 ) for determining a range to which a viewer Is likely to fix his or her eyes on; 
35 an image conversion determining step(S409) for determining a method for image conversion so that the gaze 

area becomes a region in which stereoscopy Is easy according to an output of the gaze area determining step 

(S401); and 

an image data creating step(S1 1 7) for creating the image data to be displayed on the display according to an 
output of the image conversion determining step. 

23. A storage medium for storing computer programs for displaying a pair of picture images seen from two different 
viewpoints with respect to a model on a display screen stereoscopically In computer system(5), characterized by 
comprising: 



45 a gaze range determining Instruction means(1 3) for giving Instructions for determining a range to which viewers 

are likely to pay attention to the computer system(5); 

a model moving instruction means(14) forgiving Instructions for moving the model in a direction perpendicular 
to a display screen so that the gaze range determined by the gaze range determining instruction means(1 3) 
comes in a region in which stereoscopy is easy by changing data of the modei to the computer system(5); 
so a model enlarging/reducing instruction means(15) for giving instructions for enlarging/reducing a size of the 

model moved by the model moving Instruction means(14) according to a distance over which the model is 
moved by changing data of the model changed by the model moving instruction means(14) to the computer 
system(5); and 

an image data creating instruction means(20) for giving instructions for creating image data to be displayed 
ss on the display according to data of model which has been moved by the model moving instruction means(1 4) 

and whose size has been changed by the model enlarging/reducing instruction means(15) to the computer 
system(5). 



21 



EP0 817 123 A1 



24. The storage medium according to claim 23, characterized by further comprising a depth distance reducing instruc- 
tion means(16) for giving instructions for setting a scale in the direction of depth seen from the viewer so that at 
least the major part of a model moved according to an instruction of the model moving instruction means(1 4) comes 
in the region in which stereoscopy is easy to the computer system(5). 

5 

25. The storage medium according to claim 23, characterized by further comprising a visual-line/gaze range detecting 
instruction means(30) for giving instructions for detecting viewer's visual line and gaze range with respect to an 
image actually displayed to the computer system(5), wherein the model moving instruction means(14) gives in- 
structions for moving the model In a direction perpendicular to the display according to a gaze range detected by 

TO an Instruction of the visual-llne/gaze range detecting Instruction means(30). 

26. The storage medium according to claim 23, characterized in that the model moving instruction means(1 4) includes 
a means for, when a change occurs in the placement condition of the model seen from the viewpoint and the model 
moving instruction means(14) moves the model according to the change, giving instructions for preventing an 

15 amount of moving of the model within the gaze range from exceeding about one degree in terms of angle of 

convergence for a predetermined period of time after the change in the placement condition occurs to the computer 
system(5). 

27. A storage medium for storing computer programs for displaying a model stereoscopically in computer system(5) 
20 using a pair of images seen from two different viewpoints, characterized by comprising: 

gaze range determining instruction means(1 3) for giving instructions for determining a range to which a viewer 
is likely to pay attention with respect to the model to tho computer system(5); 

a perspective transformation determining instruction mcans(32) for giving instructions for determining a per- 
25 spective transformation method of the model data to the pair of images for the left sye and the right eye 

according to an output of the gaze range determining instruction to the computer system(5); and 
an image data creating instruction means(20) for giving instructions for creating the image data to be displayed 
on the display according to a perspective transformation method determined by the perspective transformation 
determining Instruction. 

28. The storage medium according to claim 23 or 27, characterized In that the gaze range determining Instruction 
means(1 3) has a means for giving instructions for estimating a gaze range for both eyes by taking into consideration 
all or part of placement condition, tone, brightness, saturation, size, complexity and moving speed of the model 
seen from a viewpoint of a viewer to the computer system(5). 

35 

29. The storage medium according to claim 23 or 27, characterized by further comprising out-of-field data removing 
instruction means(12) for giving instructions for removing data of a model existing out of field of view to the computer 
system(5). 

40 30. The storage medium according to claim 29, characterized by further comprising a visual-line/gaze range detecting 
instruction means(30) for giving instructions for detecting viewer's visual line and gaze point with respect to an 
image actually displayed to the computer system, wherein the out-of-field removing instruction means gives in- 
structions for calculating a field of view on the basis of a visual line detected by an instruction of the visual-line/ 
gaze range detecting Instruction means and removing data existing out of the field of view to the computer system. 

45 

31. The storage medium according to claim 23 or 27 characterized by further comprising a tone changing instruction 
means for giving instructions for changing a tone of a displayed model so that the further away from the viewpoint, 
the closer the tone becomes a background tone. 

50 32. The storage medium according to claim 23 or 27, characterized in that the image data creating instruction means 
has a means for giving instructions for determining stereoscopic condition of graphic user interface items to be 
displayed on the display screen to the computer system. 

33. A storage medium for storing computer programs for displaying a pair of images seen from two different viewpoints 
55 stereoscopically in computer system, characterized by comprising: 

gaze area determining instruction means for giving instructions for determining a range to which a viewer is 
likely to pay attention to the computer system; 
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an image conversion determining instruction means for giving instructions for determining a method for Image 
conversion so that the gaze area becomes a region in which stereoscopy is easy according to an output of 
the gaze area determining instruction; and 

an image data creating instruction means for giving instructions for creating the image data to be displayed 
5 on the display according to an output of the image conversion determining means. 

34. The storage medium for storing image data, adapted to a system in which images for the left eye and right eye 
are displayed on the display screen so that the Images are viewed by the left eye and right eye separately to 
provide stereoscopy, characterized In that the storage medium Including a storage section for storing an Image 

10 conversion method for positioning a gaze area of the image In a region in which stereoscopy Is easy for each scene. 

35. The storage medium manufacturing system for manufacturing the storage medium according to claim 34, charac- 
terized by comprising: 



a gaze area determining means for determining an image portion to which the viewer is likely to pay attention, 
of image data; 

an image conversion determining means for determining a method for Image conversion so that the gaze area 
becomes the region In which stereoscopy Is easy according to an output of the gaze area determining means; 
and 

a means for describing the image conversion method In the storage medium together with the image data. 



36. A stereoscopic display system for producing im age data to be displayed on a display screen using the storage 
medium according to claim 34, characterized by further comprising a means for processing image data st ored in 
the storage medium according to the image data conversion method stored in the storage medium and gene rating 
a pair of images to be displayed on the display screen for the left eye and the right eye. 

37. The system according to claim 1 or 5, characterized by further comprising means(22) for varying a stereoscopic 
condition to diminish the stereoscopic effect. 



30 38. The system according to claim 1 or 5, characterized by further comprising means(23) for shading-off/blurring the 
edges of right and left Image of the model, where the model Is situated at the edge of the display screen. 

39. The method according to claim 12 or 16, characterized by further comprising a step of varying a stereoscopic 
condition to diminish the stereoscopic effect. 



40. The method according to claim 12 or 16, characterized by further comprising a step of shading-off/blurring the 
edges of right and left image of the model, where the model is situated at the edge of the display screen. 
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